In an automated software test environment for testing a software program, typically thousands of tests are run daily. A test can include one or more commands and expected or non-expected outputs from the performance of those commands. One or more tests can be combined and included in a test script, and execution of the program follows the test(s) laid out in the test script. Performance of a test is automated and the outcome of performing the test is either a pass or failure, determined by whether the specified expected (or non-expected) output is obtained. After performance of the tests, test failures are typically analyzed, however this is often an inefficient expenditure of resources.